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ABSTRACT 

In the course of developing a program evaluation 
scheme, a study was done of program topics identified as important to 
problem-based learning (PBL) Dy faculty and students at the John A. 
Burns School of Medicine (JABSOM) at the University of Hawaii (Manoa) 
where a PBL curriculum is used. The study interviewed and surveyed 
key stakeholders, asking about the program topics an evaluation 
should address and priorities among the suggested topics. Data were 
collected from 8 basic science and 8 clinical faculty involved in the 
planning and administration of the curriculum and 10 students. In 
addition, about 40 first-year students and 25 second-year students 
participated in separate group interviews. The results were content 
analyzed and summarized into 43 program topics in 9 categories. A 
socond survey was done to collect responses on how the topics should 
be prioritized. This process revealed some differences between 
faculty and students in ranking of topics and eventually resulted in 
the following ranking for the top half of the scale (51-100): (1) 
tenets of the program; (2) understanding and practice of PBL; (3) 
knowledge and skills; (4) competent, caring, ethical, and 
well-adjusted physicians; (5) psychosocial and population knowledge; 
(6) congruence and consistency of curricular methods; (7) affective 
environment; (8) studont assessment; and (9) teaching environment. 
Included are 1 table, an appendix containing results, and 16 
references. ( JB) 
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Program Topics Identified by Faculty and Students as Important 
for Program Evaluation in a Problem-Based Medical School Curriculum 



Introduction 

Over the years, traditional medical education has faced some difficult 
problems, among them (a) a body of knowledge that students cannot cover in 
four years of medical school, (b) the rapid obsolescence of parts of this 
knowledge, (c) too great a focus on teaching and too little on learning, and (d) 
a lack of attention to social and psychological issues and their effects on health 
care. To help address these problems, the John A. Burns School of Medicine 
(JABSOM) at the University of Hawai'i at MSnoa has joined the growing list 
of medical schools that have implemented problem-based learning (PBL) 
curricula (Barrows & Tamblyn, 1980; Boud & Feletti, 1991). JABSOM was 
one of the first medical schools in the world to fully adopt PBL over all four 
years without an alternative traditional curriculum track. The JABSOM PBL 
curriculum is designed to (a) reduce information overload by helping students 
define what they need to know, (b) emphasize self-directed learning, (c) 
organize medical concepts and facts within the context of clinical problems 
rather than by disciplines, (d) emphasize humanistic and behavioral under- 
standing and skills, and (e) provide students with extensive experience in 
community-based medical education. Unlike their predecessors in JABSOM's 
traditional curriculum, students in the PBL program have gathered in small 
groups of five or six where, guided by faculty tutors, they have actively 
directed their own learning. 

In the second year of the PBL program, JABSOM contracted Curriculum 
Research and Development Group (CRDG) of the University of Hawai'i at 
Manoa to design and conduct an evaluation of the PBL curriculum. When 
developing the evaluation design, CRDG interviewed and surveyed key PBL 
stakeholders, including clinical faculty, basic science faculty, and students, and 
asked about (a) the program topics the evaluation should address and (b) 
priorities among these topics. The results of the interviews and survey, 
including information about the content of the program topics and between- 
group differences in topic priority, have implications not only for evaluation 
but also for prog am development. In this paper, the methods of the study 
are described and the implications of the study's results for program 
evaluation and program development are presented. 

Background 

It has long been recommended that program-evaluation stakeholders 1 be 
involved in the design and implementation of program evaluation studies. In 
recent years, such involvement has become increasingly frequent. Involving 



'Shadish, Cook, & Leviton (1991) have defined program-evaluation stakeholders as 
•those whose lives are affected by the program and its evaluation* (p. 179). 
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stakeholders in the design of evaluations improves the studies' validity 
(Brandon & Newton, 1992; Owlston, 1986) and the potential for the utilization 
of evaluation findings (Greene 1988a, 1988b; Weiss, 1983; Mark & Shotland, 
1985). However, with the exception of the evaluation reported by Kerbeshian 
(1986), medical schools have not sought input from faculty and students in the 
design of curriculum evaluations. Consequently, they have obtained little 
information from stakeholder groups who are affected by the curriculum and 
who have considerable first-hand knowledge about it. Some issues that may 
be important to curriculum participants, such as concerns about the implemen- 
tation of the program, may not have been adequately addressed by these 
evaluations. With few exceptions (e.g., Cleeve-Hogg & Byrne, 1988; Gijselaers 
& Schmidt, 1990), evaluations of innovative medical school curricula have 
examined only student outcomes (see, for example, Schmidt, Dauphinee, & 
Patel, 1987), an approach recommended by some leading PBL evaluators 
(Friedman et al., 1990). 

Data collected from stakeholders during the design of evaluation studies 
may provide information that can be used in ongoing program development. 
Stakeholders' input can inform curriculum planners about the extent of 
participants' understanding of the curriculum, the parts of the curriculum they 
think important, and the concerns they have about its implementation and 
outcomes. Curriculum planners and administrators can use this information 
to (a) examine congruence between the goals of the curriculum and the goals 
of its participants, (b) identify problems in the implementation of the 
curriculum, (c) identify features or components of the curriculum that require 
greater elaboration or emphasis, and (d) plan future educational activities, 
such as tutor training workshops and student orientations. Stakeholder 
participation in the design of evaluation studies should provide greater clarity, 
strengthen the understanding and increase the involvement of all who are 
associated with the curriculum, and enhance the commitment to curriculum 
goals and methods. 

Methods for Identifying and Prioritizing Program Topics 
Identifying the Topics 

For this study we defined stakeholders as JABSOM's clinical faculty, basic 
science faculty, and students. Our first task was to identify program topics. 
In open-ended interviews, faculty and students were asked (among other 
things) about the program topics they would like the evaluation to address. 
Individual interviews were conducted with (a) eight basic-science and eight 
clinical faculty involved in the planning and administration of the PBL curricu- 
lum and (d) 10 students. About 40 first-year students and 25 second-year 
students (the first two cohorts in this curriculum) participated in separate 
group interviews. Our rationale for involving students was to enhance the 
validity of the findings— that is, to obtain information about the curriculum 
that might be unknown to the faculty or program planners. 



Following the conclusion of the interviews, the results were content ana- 
lyzed and summarized into 43 program topics. These topics, shown in 
Appendix A, fall into nine categories: (a) tenets of PBL (three topics); (b) 
understanding and practice of PBL (two topics); (c) knowledge and skills 
(seven topics); (d) competent, caring, ethical, and well-adjusted physicians (six 
topics); (e) psychosocial and population knowledge (five topics); (f) congru- 
ence and consistency of curricular methods (nine topics); (g) affective environ- 
ment (seven topics); (h) student assessment (three topics); and (i) teaching 
environment (one topic). 
Prioritizing the Topics 

The second task in the study was to prioritize the topics according to their 
importance to the evaluation. A questionnaire, in which the 43 topics were 
presented in randon »rder, was developed, pilot-tested, and distributed to the 
three groups of stakeholders: (a) clinical faculty who were familiar with the 
PBL curriculum (N= 153); (b) basic science faculty who were familiar with the 
PBL curriculum (7V=41); and (c) students participating in the PBL curriculum 
(7V=125). The questionnaire used the magnitude-scaling method (Lodge, 1987; 
Witkin, 1984). If certain assumptions are met, magnitude-scale scores are on 
a ratio scale and can be interpreted as ratios of one to another. Respondents 
were asked to indicate the importance of the program topics by assigning each 
topic a value relative to a reference topic, which had a preassigned value of 
100. Because all the topics represented issues important to some program 
participants, the scaling procedure required that they make decisions about 
the relative importance of each topic. Response rates were 42.5% (n=65) for 
the clinical faculty, 70.7% (n=29) for the basic science faculty, and 59.2% 
(n=74) for the students. 2 

The raw data were analyzed as follows (Lodge, 1987): (a) the data were 
transformed to logs with base 10; (b) the arithmetic mean of the logs was 
calculated for each of the 43 topics; and (c) the geometric mean of each 
arithmetic mean was calculated (that is, the arithmetic mean was raised to the 
10th power). For the purposes of this paper, the questionnaire data were 
scaled separately for clinical faculty, basic science faculty, and students. Each 
of the three sets of scale scores was then linearly transformed to a common 
scale. For each program topic, the transformed score was calculated as: 

score,, k _ 

topicscoreM *—) 100, 

maxscorej 

where score y=the magnitude scale score on topic i for group ; and max- 
scorej-Xht highest of the magnitude scale scores for the group. This 



^ese response rates differ from a companion paper (Brandon, Lindberg, & Wang, 1992) 
because the results reported here include data from late respondents. 



transformation resulted in three sets of scores scaled from 0 to 100 3 , as shown 
in Appendix A. 

Results of the Identification and Prioritization of Program Topics 
In Table 1, the topics in the top half of the scale (51-100) are shown for 
each respondent group. Table 1 is summarized as follows. 



Insert Table 1 about here 



Category 1: Tenets ofPBL (Three Topics) 

The topic, self-directed, life-long learning, was the highest scaled topic for 
all three groups (see Appendix A). The topic, thinking and reasoning skills, also 
was in the top half of the scale for all three groups, while the topic, student- 
centered learning, was in the top half for basic science faculty and students, but 
not for clinical faculty. 

Category 2- Understanding and Practice of PEL (Two Topics) 

Students scaled the topic, student and faculty (a) understanding of PBL 
objectives and (b) participation in PBL, in the top half of all topics. 
Category 3: Knowledge and Skills (Seven Topics). 

For Category 3, the clinical faculty scaled only one topic, clinical knowledge 
and skills, in the top half of the scale. The basic science faculty and students 
each scaled four topics in the top half. For both groups, these topics included 
basic science knowledge, teamwork and communication skilb, and clinical know- 
ledge & skills; for the basic science faculty, the fourth topic was continuation 
of basic science through the clinical years, and for the students, the fourth topic 
was integrating information from different disciplines. 
Category 4: Competent, Caring, Ethical, and Wett-Adjusted Physicians (Six 
Topics) 

All three groups of respondents scaled the topic, competent and caring 
practitioners, in the top half of the scale. 
Category 5: Psychosocial and Population Knowledge (Five Topics) 

For this category, clinical and basic science faculty included the topic, 
philosophical and community-oriented perspective, in the top half of the scale; 
students included none. 

Category 6: Congruence and Consistency of Curricular Methods (Nine Topics) 
All three groups scaled the topic, curriculum coverage, in the top half of the 



'The scores of 100 show the highest group scores, and the scores of 0 indicate the zero 
points on the original magnitude scales. 

4 



scale. In petition, the students scaled the topic, mode of learning, in the top 
half of their results. 

Category 7: Affective Environment (Seven Topics) 

Neither faculty group included any topics from this category in the top half 
of their results. The students included four: communication; noncompetitive, 
positive environment; commitment and respect; and advisor support for students. 
Category 8: Student Assessment (Three Topics). 

For this category, neither faculty group scaled any topics in the top half of 
their results. The students, however, included two topics in the top half of the 
scale: tracking student performance and review of the effects of external 
influences. 

Category 9: Teaching Environment (One Topic). 

None of the three groups placed the topic in this category in the top half 
of the scales. 

Implications of the Findings for Curriculum Evaluation 
Stakeholder-based evaluations have been recommended for involving 
"people holding different positions in the social structure of programs" 
(Weiss, 1983, p. 8). Stakeholders have knowledge and experience that enable 
them to specify program topics that can enhance the understanding of a 
program (Mark & Shotland, 1985). In the study reported here, the stakehol- 
ders identified topics suggesting a greater breadth of program evaluation than 
is currently common among medical schools offering PBL curricula. In 
addition to topics commonly selected for evaluation of medical-school PBL 
curricula, such as those included in the category, Knowledge and Skills, 
JABSOM faculty and students identified topics in categories such as 
Congruence and Consistency of Curricular Methods and Understanding and 
Practice of PBL. 

The differences in priorities among the JABSOM stakeholder groups 
suggest the appropriateness of giving students a say in identifying the program 
topics an evaluation might address. 4 Of the 17 topics in the top half of the 
students' scale, 7 were not included in the top half of either faculty group's 
scale. The students' three high-scaled topics , in the category, Affective 
Environment, probably would not have been included if the faculty alone had 
been responsible for identifying the topics. Partly due to its identification in 
this study, one of these three topics {communication) is currently the focus of 
considerable program effort. Examples such as these show how the 
involvement of the stakeholder groups enhances the validity of the program 
topics. 



Stakeholder involvement in topic selection that results in differences among stakeholder 
groups' topic priorities, such as in the study reported here, requires evaluators to take 
additional steps to reconcile differences between groups. The steps we took are presented in 
Brandon, Lindberg, and Wang (1992). 

5 
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Some of the topics in the top half of the scale for one or more of the three 
stakeholder groups (see Appendix A) suggest evaluation designs and methods 
other than the typical medical-school PBL studies concentrating solely on 
student outcomes. Topic 6A, curriculum coverage, for example, which was 
scaled high for all three groups, clearly requires an examination of curriculum 
operations. Similarly, two of the topics in the category, Affective Environment, 
show high scores for the students (communication and noncompetitive, positive 
environment), indicating a need to examine curriculum implementation. Other 
topics such as self-directed, life-long learning and thinking & reasoning skills 
(scored very high by all three groups) suggest evaluations of both program 
implementation and outcomes. 

Implication of the Findings for Curriculum Development 
The findings presented here reflect JABSOM participants' understanding 
of the curriculum, the extent to which their values and expectations are 
consistent with the goals of the curriculum, and the influence of the unique 
circumstance and culture of each group on the values and expectations of its 
members. Stakeholder knowledge and values are to some extent interdepen- 
dent; together, they determine much of what actually happens in the 
educational setting. The challenge that confronts curriculum developers, then, 
is to ensure that all groups, each with its different knowledge base and values, 
understands, acce ls, and implements the new curriculum. 

To help meet this challenge, we have examined some of the findings we 
thought most salient for curriculum development. These findings are based 
on the importance ascribed by the three groups to topics within the categories, 
Tenets of PBL, Knowledge and Skills, Psychosocial and Population Knowledge, 
Congruence and Consistency of Curriculum Methods, and Affective Environment. 
Tenets of PBL 

The category, Tenets of PBL, is clearly the most important for all three 
groups. The high value attributed to self-directed, life-long learning, and 
thinking and reasoning skills indicates success in communicating to participants 
how students are to learn in the PBL curriculum. The low score assigned by 
the clinical faculty to student-centered learning, however, suggests that clinical 
faculty as a group did not value this goal. These findings reflect the 
discrepancy in training received by the students and basic science faculty, who 
have spent a great deal of time in close proximity to the core group of 
curriculum developers at the medical school, and clinical faculty, many of 
whom have private practices in the community and have spent little time at 
the school. In the first two years of the program, tutor training workshops, 
required of faculty who chose to participate as tutors in the PBL educational 
process, were attended largely by bosic science faculty. Clinical faculty who 
elected to serve as clinical skill preceptors, however, often did not attend the 
orientations and follow-up meetings scheduled to prepare them to take part 
in PBL. 



• 



The findings reinforce the need for systematic communication of 
curriculum philosophy, goals, and process to all faculty. The curriculum has, 
in fact, responded to this need. For example, clinical chairs have encouraged 
more of their faculty to attend tutor training workshops and program planners 
have more carefully monitored participation in preceptors' meetings. Clinical 
faculty have responded positively; an upcoming tutor training workshop, 
specifically for faculty from clinical departments, has been oversubscribed by 
both compensated and volunteer clinical teachers. 
Knowledge and Stalls 

It is in this category of topics that the influence of cultural differences on 
group values is most apparent. Students placed a high value on acquisition 
of both clinical knowledge and skills and basic science knowledge, as well as on 
integrating information from different disciplines, Both faculty groups, how jver, 
valued acquisition of their respective knowledge more highly than they valued 
acquisition of knowledge of the other group. Neither faculty group scored 
integrating knowledge from different disciplines as highly as students. 

This result is informative but not surprising, for such divergence is likely 
to be found among groups within any profession. Perhaps integration of 
knowledge will be more highly valued as the curriculum, which has de-empha- 
sized the traditional distinction between preclinical and clinical years and 
encouraged integration of basic science and clinical learning experiences, 
becomes more fully implemented. Basic science learning experiences are 
currently being built into the third and fourth years of the curriculum, just as 
clinical experiences have been built into the first and second years. Students 
will study both biological and clinical sciences over all four years. 
Psychosocial and Population Knowledge 

Just one topic in this category addresses psychosocial issues. Although 
both faculty groups gave psychosocial aspects of medicine higher scores than 
the students, no group placed this topic in the top half of the scale. The 
remaining four topics address the population perspective of health and illness 
and community medicine— two closely related issues. The importance assigned 
to the four topics varies widely among them. The most probable explanation 
for this variation is that the participants responded to different aspects of the 
four statements. In general, faculty attributed greater importance to all four 
topics than did students. Thus, one reasonable conclusion about the results 
for this category is that JABSOM faculty value both psychosocial and 
population perspectives of medicine more highly than do the students. 

JABSOM's curriculum goals emphasize acquisition of psychosocial and 
population knowledge as well as a community orientation toward medicine. 
JABSOM students, focused on mastering basic science and clinical knowledge, 
do not yet consider pop ulation and psychosocial aspects of health care a high 
priority. The learning gcals of students might be brought into closer alignment 
with the curriculum's goais if (a) greater emphasis were placed on these 
knowledge perspectives in student orientations and faculty training and (b) the 

7 
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prominence of these perspectives in learning materials and evaluation of 

student performance were increased. 

Congruence and Consistency of Curriculum Methods 

All groups considered curriculum coverage an important program topic. 
Students also included mode of learning among the topics they thought most 
important. 

The curriculum has addressed the problem of rapid growth and change in 
medical knowledge by encouraging commitment to life-long learning and 
discouraging the belief that students can learn everything they need to know 
to become competent physicians in their four years of medical school. Never- 
theless, both faculty and students are concerned about the comprehensiveness 
and balance of the material covered by the curriculum and the match between 
this material and curriculum objectives. 

Part of the concern can be traced to the abundance of medical knowledge 
available to students and to their uncertainty about how much of this 
knowledge they need to acquire. Unlike their counterparts in traditional, 
teacher-centered curricula, PBL students decide for themselves what and how 
much they must learn. Accustomed to having information presented in 
measured doses by their college professors, students may find this a formida- 
ble responsibility. Faculty, accustomed to deciding how much information 
from their particular disciplines should be transmitted to students, may also 
feel uncertain about the depth and breadth of coverage of information in the 
interdisciplinary PBL setting. 

Another part of the concern about curriculum coverage arises from actual 
gaps and overlaps in the material covered in tutorials, preceptorships, and 
other learning contexts, as well as the match between the level of students' 
understanding and the level at which information is presented to them in 
colloquia and conferences. 

It is likely that students' concern with mode of learning is in part related to 
these curriculum coverage issues and to the anxiety engendered by the newly 
acquired responsibilities that attend student-centered learning. Most students 
have not encountered these responsibilities in an educational setting before 
entering the PBL curriculum. 

Concerns about curriculum coverage can be addressed in several ways. 
One approach would be to (a) identify the basic science and clinical informa- 
tion that must be learned and (b) develop an order of presentation of this 
information that best promotes integration of new and prior knowledge. A 
somewhat less laborious approach would be to systematically examine, or 
track, the curriculum. Curriculum tracking would uncover the areas in which 
information is omitted or repeated, as well as the areas in which one discipline 
is overemphasized at the expense of another. The school is currently taking 
the initial steps to develop a tracking system. 

Concerns about curriculum coverage might also be lessened by additional 
training of both faculty and students. Curriculum participants, new to this 
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learning format, can increase their skills and confidence through discussion, 
practice, and evaluation. 
Affective Environment 

The striking difference between faculty and students in values assigned to 
topics in this category dearly icflects the difference in their respective 
circumstances within the medical school. Clinical faculty judged attitudes 
toward the school to be somewhat important, but neither faculty group 
considered significant any other topic addressing the affective environment. 
Students, on the other hand, attached great importance to four topics 
addressing interpersonal relations: communication; noncompetitive, positive 
environment; commitment and respect; and advisor support for students. All but 
advisor support for students were among the students' 10 highest scaled topics. 

These findings clearly indicate that for students interpersonal relations are 
a critical part of the medical school experience, JABSOM curriculum 
planners have responded to a number of student concerns about interpersonal 
relations, including communication between faculty and students. A student 
representative from each class now participates in weekly meetings of the 
school's committee charged with making programiratic decisions. Students 
are represented on a number of subcommittees addressing focused curriculum 
development and implementation issues. As committee members, they both 
receive and give information in a timely fashion. Thus, a constructive 
mechanism for the mutual exchange of information between curriculum 
developers and students has been established. This mechanism will allow 
students to discuss with curriculum developers all concerns related to the 
affective environment and learning experiences. 

Conclusions 

The findings of the study reported here provide information about 
stakeholders' knowledge, values, and expectations about a medical-school PBL 
curriculum at the end of its second year of implementation. 3 Much of this 
information is not new to JABSOM curriculum developers and may be 
familiar to developers of other established PBL curricula. To our knowledge, 
however, never before has information on PBL program topics been collected, 
summarized, and presented in such a systematic fashion. It also is the first 
time it has been prioritized, thereby informing evaluators and planners of the 
relative value attached to a variety of curriculum issues by the three 
stakeholder groups. 

The findings reported here have implications for evaluators of PBL 
programs. First, the identified program topics suggest a greater breadth of 
program evaluation than is commonly found at medical schools offering PBL 
curricula. Some of the high-priority topics discussed here could not be 



^The curriculum has continued to evolve since this study was conducted. Therefore, some 
of information reported here may currently be somewhat less applicable. 
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covered in an evaluation that examines only student outcomes. Evaluators of 
PBL programs might consider focusing on a wide variety of topics or consider 
using a variety of approaches, including not only the study of program 
outcomes but also of program implementation and the relationship between 
implementation and outcomes. Seconds the differences in priorities among 
JABSOM stakeholder groups suggest the appropriateness of including students 
as stakeholders. Such differences provide additional support to the notion 
that program beneficiaries should help identify the topics that program 
evaluations might address. 

Our findings suggest at least three implications for PBL-curriculum 
developers in the early stages of their curricula. First, administrators might 
consider how to address the difficulties of involving part-time clinical faculty 
in a radically new program such as PBL. For example, JABSOM clinical 
faculty assigned a low value to student-centered learning, a key tenet of PBL, 
suggesting that it may have been less difficult to reach basic science faculty 
than clinical faculty in the early stages of the program. At JABSOM, 
administrators have taken steps to increase the enrollment of clinical faculty 
in tutor training workshops and have more carefully monitored participation 
in clinical preceptors' meetings. 

Second, differences in priorities that stakeholder groups give to program 
topics show variations in the academic culture and in group values that affect 
the progress and operation of a new curriculum such as PBL Among 
JABSOM faculty, for example, both the clinical and basic science groups 
assigned high priorities to their respective fields of knowledge and gave lower 
priority to the fields of their faculty colleagues. Neither faculty group valued 
the topic, integrating knowledge from different disciplines, as highly as the 
students. Students, on the other hand, gave high priority to the curriculum's 
mode of learning, showing concern over the responsibilities of student-centered 
learning, while faculty scored this topic low. Similarly, the students thought 
that interpersonal relations are a critical part of the medical school experi- 
ence; they scaled topics about the affective environment higher than the 
faculty. Such differences may be common in many educational programs, 
particularly in their early phases. However, as we are rinding, such differences 
should lessen as programs develop and mature. At JABSOM, we expect that 
the faculty will continue to increase the integration of basic science and clinical 
learning experiences. For students, newly-established student-faculty 
communication mechanisms and student participation on JABSOM curricular 
planning committees have provided them a larger role in planning and 
decision making. 

Third, the comprehensiveness and balance of the material covered by a 
new curriculum, and the match between this material and curriculum 
objectives, are two potential issuss for planners of new PBL programs. In an 
innovative curriculum, students will have new, unfamiliar responsibilities for 
deciding what and how much to learn, and faculty may lack confidence about 
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the depth and breadth of coverage of information in interdisciplinary PBL 
settings. Both groups may be cancans d about gaps and overlaps in the 
material covered in tutorials, preceptorships, and other learning contexts. As 
JABSOM is doing, PBL planners might consider developing curriculum 
tracking systems for identifying Lie information which is omitted, repeated, or 
overemphas < . > Additional training of both faculty and students might also 
help addre' o« h groups' concerns. 
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Table L Topics in the Top Half (51-100) of the Scale for Each Group 



Category of topic 


Topic* 


Clinical faculty 


Basic science faculty 


Students 


1. Tenets of PBL (3 to- 
pics total) 


1A: self-directed, life-long 
learning; 

18: thinking & reasoning 
skills. 


1A: self-directed, life-long 
learning; IB: thinking & 
reasoning skills; 1C. stu- 
dent-centered learning. 


1A: self-directed, life-ion 
g learning; IB: thinking 
& reasoning skills; lC: 
student-centered learn- 
ing. 


2. Understanding and 
Practice of PBL (2 topics 
total) 


None 


None 


2A: student & faculty (a) 
understanding of PBL 
objectives and (b) parti- 
cipation in PBL. 


3. Knowledge & Skills (7 
topics total) 


3A: clinical knowledge & 
skills. 


3C: basic science kn™"ied 
ge; 3F: continuation of ba 
sic science through the 
clinical years; 3B: team- 
work & communication; 
3A: clinical knowledge & 
skills. 


3A: clinical knowledge & 
skills; 3C: basic science k 
nowledge; 3D: integrating 
information from differ- 
ent disciplines; 3B: team- 
work & communication 
skills. 


4. Competent, Caring, 
Ethical, and Well-Adjuste 
d Physicians (6 topics 
total) 


4A: competent & caring 
practitioners. 


4A: competent & caring 
practitioners. 


4A: competent & caring 
practitioners. 


5. Psychosocial and Popul 
ation Knowledge (5 to- 
pics total) 


SA: philosophical & com 
munity-oriented perspec- 
tive. 


5A: philosophical & com 
munity-oriented perspec- 
tive. 


None 


6. Congruence and Consi 
stency of Curricular 
Methods (9 topics total) 


6A: curriculum coverage. 


6A: curriculum coverage. 


6A: curriculum coverage; 
6D: mode of learning. 


7. Affective Environment 
(7 topics total) 


None 


None 


7C: communication; 7E: 
noncompetitive, positive 
environment; 7B: com- 
mitment & respect; 7D: a 
dvisor support for stu- 
dents. 


& Student Assessment (3 
topics total) 


None 


None 


8A: tracking student perf 
ormance; 8C: review of ef 
fee's of external influ- 
ences. 


9. Teaching Environment 
(1 topic) 


None 


None 


None 



"The topic numbers shown in these columns are from Appendix A, 
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APPENDIX A 
SURVEY RESULTS ON THE IMPORTANCE OF 
PROGRAM TOPICS, BY GROUP AND CATEGORY OF TOPIC 



Program topic 


Clinical 
faculty 


Basic 
science 
faculty 


Students 


Category l: Tenets of PEL 


1A. Self-directed, life-long learning (preparation of students to become 
independent, self-directed, life-long learners.) 


100.00 


100.00 


100.00 


IB. Thinking & reasoning skills (development of thinking and reason- 
ing skills such as the generation of hypotheses and the identification 
and critical appraisal of necessary information, and problem-solving). 


95.79 


98.47 


81.23 


1C. Student-centered learning fosters student-centered, rather than 
teacher-centered, learning.) 


26.13 


65.13 


53.95 


Category 2* Understanding and Practice of PBL 


2A. Student & faculty (a) understanding of PBL objectives and (b) 
participation in PBL (studen:, faculty, and staff understanding of PBL 
objectives and their respective roles in the program; encouragement 
of participation in the PBL process) 


35.37 


44.07 


65.73 


2B. Applicants' understanding of PBL (Medical School applicants' 
understanding of the PBL philosophy and process before they are 
accepted into the program) 


8.71 


21.98 


42.07 


Category 3: Knowledge and Skills 


3A. Clinical knowledge and skills (acquisition of adequate clinical kno 
wledge and skill in applying this knowledge to patient care) 


69.48 


51.70 


70.41 


3B. Teamwork & communications skills (development of teamwork 
and of skill in communicating with other health care workers and 
with patients) 


45.25 


52.90 


54.40 


3C. Basic science knowledge (acquisition of adequate basic science 
knowledge) 


38.29 


78.43 


62.37 


3D. Integrating information pom different disciplines (integration and s 
ynthesis of information from different disciplines and knowledge per- 
spectives) 


35.25 


43.01 


54.66 


3E. Learning resources (provision of adequate and varied learning re- 
sources and facilitation of their use) 


29.93 


34.90 


28.91 


3F. Continuation of basic science through the clinical years (continua- 
tion uf the basic sciences into the 3rd and 4th year of medical school) 


25.69 


53.07 


16.66 


3G. Preparation for the NBME (preparation of students to pass the 
NBME examinations) 


19.50 


23.93 


46.87 



14 



(APPENDIX A, continued) 



Program topic 


Clinical 
faculty 


Basic 
science 
faculty 


Students 


Category 4: Competent, Caring, Ethical, and Well-Adjusted Physicians 


4A Competent & caring practitioners (development of competent, 
caring, and ethical practitioners who demonstrate humility in their 
relationships with other health care workers, patients, and their fami- 
lies) 


93.83 


63.05 


91.53 


4B. Satisfied physicians (production of physicians who are satisfied 
with their career choices; fostering of enthusiasm for the profession 
of medicine"} 


47.12 


32.55 


47.68 


4C. Student learning about self (promotion of student learning about 
their habits, attitudes, and values; development of the habits, atti- 
tudes, and values that are expected of physicians) 


45.63 


43.59 


30.58 


4D. Student emotional well-being (fostering of emotional health and 
development of self-esteem in students) 


40.87 


45.13 


36.67 


4E. Social awareness & civic involvement (development of social 
awareness and involvement in civic activities) 


19.88 


16.50 


18.35 


4F. Physicians' emotional health (effect of educational experiences on 
physicians' emotional health and effect of emotional health on qualit 
y of patient care) 


19.85 


12.95 


22.08 


Category 5: Psychosocial and Population Knowledge 


5A Philosophical & community-oriented perspective (acquisition of a 
broad, philosophical— not just technical and clinical— perspective of 
medicine and decision-making that is in the best interest of the 
community as well as the individual patient) 


58.43 


56.27 


44.89 


SB. Psychosocial aspects of medicine (development of an understand- 
ing of the psychosocial aspects of health and illness) 


46.45 


41.14 


32.13 


SC. Understanding community medicine <& promoting wellness (devel- 
opment of an understanding of the role of community medicine and 
the value of promoting wellness and preventing illness) 


37.83 


41.07 


35.29 


5D. Population perspective (acquisition of a population perspective of 
health and illness) 


23.97 


24.16 


21.49 


5E. Community-oriented, primary care (development of an interest in 
community-oriented, primary care practice) 


22.39 


28.94 


17.05 
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(APPENDIX A, continued) 



Program topic 


Clinical 
faculty 


Basic 
science 
faculty 


Students 


Category &- Congruence and Consistency of Cumadar Methods 


6A. Curriculum coverage (provision of health care problems that re- 
flect program objectives and of comprehensive and balanced coverage 
of basic and clinical sciences) 


56.65 


63.12 


73.29 


6B. Specification of clinical skills to be covered and consistency among 
oreceotors fsnecification of sets of clinical skills to be covered in the 
units; consistency among preceptors in the kinds of learning activities 
they provide students and in their expectations of students) 


31.13 


30.44 


43.99 


AC Match hptwppn Ip amino activities & objectives t match between the 

student learning activities and the PBL goals and objectives) 


30.61 


40.49 


31.42 


6D. Mode of learning forovision of multiole modes of learning, in- 
eluding lecture; student participation in selection of learning mode) 


29.29 


32.72 


66.37 


6E Continuity of learning activities fcontinuitv of learning activities 
from unit to unit) 


27.21 


39.23 


33.63 


6F. Coordination of learning activities (coordination of learning 
activities between tutors and clinical skills preceptors) 


23.33 


20.86 


30.52 


6G. Opportunities for research &. for providing community education 
(provision of opportunities for students to take part in research and 
to provide community education) 


20.64 


20.55 


17.56 


6H. Consistency among tutors (consistency among tutors in the appli- 
cation of the tutorial process) 


15.58 


13.39 


26.40 


61. Interest in academic medicine & research (development of an 
interest in academic medicine and medical research) 


12.19 


15.06 


8.05 


Category 7: Affective Environment 


7A. Attitudes toward the school (fostering of enthusiastic, joyful learn- 
ing and positive attitudes toward the program and school) 


49.69 


21.30 


35.86 


70. Commitment and respect (encouragement for faculty commitment 
to students as well as to the program; development of mutual respect 
between students and their teachers) 


34.99 


28.97 


69.68 


7C. Communication (encouragement for communication among pro- 
gram planners, faculty, and students and for constructiveness of re- 
sponses to input from each other) 


33.12 


34.97 


89.53 


ZD. Advisor support for students (preparation and encouragement of 
advisors to support their students and keep them informed of 
m? iters relevant to their educational interests) 


26.13 


17.11 


53.47 
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(APPENDIX A, continued) 



Program topic 


Clinical 
faculty 


Basic 

science 
faculty 


Students 


7E. Noncompetitive, positive environment (a noncompetitive and non- 
threatening learning environment; positive interaction and resolution 
of conflict among program participants) 


26.02 


27.71 


81.85 


7F. Student involvement in decision-making (involvement of students 
in programmatic decision-making) 


19.50 


12.54 


43.22 


7G. Examination of student stress (acknowledgement of the stress 
experienced by first and second year students and examination of its 
sources) 


16.11 


26.79 


34.56 


Category 8: Student Assessment 


8A. Tracking student performance (tracking of student performance 
across all units; provision of feedback to students about their perfor- 
mance) 


39.23 


40.56 


56.80 


8B. Review of student assessment instruments (review of assessment 
instruments to see if they are measuring student performance on 
curriculum goals and objectives) 


37.65 


44.44 


47.18 


8C. Review of the effects of external influences (review of the effect of 
external influences, such as the NBME examinations, on the curric- 
ulum and on student learning) 


15.05 


15.41 


52.82 


Category 9: Teaching Environment 


9A. Faculty workload & morale (maintenance of manageable faculty w 
orkloads; recognition and reward for faculty participation in the PBL 
curriculum; maintenance of faculty morale) 


25.31 


47.55 


32.38 
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